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Office Action Summary 


Application No. 

09/430,234 


Applicant(s) 
LIENHART ET AL 


Examiner 

BRIAN- P. YENKE 


Art Unit 

2614 





Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment See 37 CFR 1 .704(b). 

Status 

1 )K Responsive to communication(s) filed on Amendment/RCE (31 Jan 05) . 
2a)D This action is FINAL. 2b)E3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-19 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) IEI Claim(s) 1^19 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) I3 Notice of References Cited (PTO-892) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) O Information Disclosure Statement(s) (PTO-1 449 or PTO/SB/08) 

Paper No(s)/Mail Date . 



4) Q Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) O Notice of Informal Patent Application (PTO-1 52) 

6) □ Other: . 
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DETAILED ACTION 

1 . Applicant's arguments with respect to claims 1-19 have been considered but are 
not persuasive. 

Applicant's Arguments 

a) Applicant states that neither Newman nor Liaguno discloses organizing 
images or associated text as hierarchical shot clusters. 

Examiner's Response 

b) The examiner disagrees. In considering the combination of Newman and 
Liaguno, Newman discloses everything except converting the annotations into 
searchable annotations using a voice-to-text conversion system, which was taught by 
Liaguno. Regarding the organization of such annotations, as stated below in the 
rejection Newman already categorizes/ranks/groups the media based upon the 
event/scene. 

It is also noted by the examiner that the rejection below was based upon the 
broadest interpretation of the claimed invention. Thus although, the applicant states in 
the specification that hierarchical pertains to the duration of the event, the claims do not 
state/reflect as such, thus a rejection is being provided as shown below. 

Continued Examination Under 37 CFR 1.114 

2. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 



Application/Control Number: 09/430,234 Page 3 

Art Unit: 2614 

application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 31 Jan 
05 has been entered. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-19 are rejected under 35 U.S.C as being unpatentable over Newman et 
al., US 6,154,600 in view of Liaguno et al., US 5,729,741 . 

In considering claims 1, 6, 10 and 16, 
a-c) the claimed receiving a video signal is met by the graphical user interface (GUI) 
(Fig 14) where the GUI can capture a variety of sources to include, a camcorder 586, 
VCR 588, auxiliary audio TV, CD-ROM/DVD and digital camera 590. 
b) the claimed receiving a first audio signal containing annotations is met where the 
graphical user interface can capture additional audio via auxiliary audio (capture 584) 
and also add audio video via storyboard 592, which allows edit, playback and record 
shots, including audio shots (col 14, line 18-22). 

However, Newman does not specifically disclose converting the annotations into 
searchable annotations using a voice-to-text conversion system. 



Application/Control Number: 09/430,234 Page 4 

Art Unit: 2614 

Newman does disclose a system that can manipulate hypermedia, where 
hypermedia refers to the integration of text, graphics, sound, video and other data, or 
any combination into a system which can be stored and retrieved (col 5, line 47-52). 
Newman also discloses that audio is synchronized to video by reference to a master 
pixel clock (col 18, line 57-59). Regarding the annotations organized as hierarchical 
shot clusters, Newman does disclose ranking/grouping (i.e. hierarchical) the media 
based upon the event/scene (i.e. GreekCity, Forest, Horse, Lightening etc) (see Fig 9). 
Newman also discloses a system which lets the user input the type of information, i.e. 
"OUR SUMMER VACATION", which is displayed in the center of the window 408 (Fig 
12). Thus given the broadest interpretation of the claimed invention "hierarchical" is 
met by Newman which discloses grouping the media based upon the event/scene. 

The conversion of voice (audio, sound) to text, where the converted information 
is searchable is conventional in the art. 

The examiner incorporates Liaguno which discloses a system for storage and 
retrieval of text, pictures, video and audio where the source of information (text, 
pictures, video and audio) are converted to text. Thus Liaguno provides a system 
where a user is able to search (via text) the desired source of information, in order to 
provide the user rapid retrieval of any type of data (col 2, lines 32-42). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Newman which discloses a video home editing system, 
which can edit/cut/insert/reproduce hypermedia including graphics, text and audio 
annotations (col 14, line 17-19), with Liaguno, to also convert the audio to searchable 
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text, in order to provide the user the ability to retrieve a desired segment, whether text, 
video, audio etc... of the hypermedia, which would provide the user rapid retrieval of a 
selected segment. 

In considering claims 2, 7, 11 and 17, 

However, neither Newman nor Liaguno disclose removing the annotations from 
the second audio signal (signal 120). 

Newman discloses a media editor where the user is able to 
add/insert/delete/remove hypermedia to an existing hypermedia file by using graphical 
user interface 400 (Fig 9, Fig 14). 

Thus based on the type of hypermedia being edited (i.e. home video), the user 
could create a hypermedia (home video) where the scene/background audio (2 nd audio 
signal) is removed in order to provide additional/auxiliary audio in lieu of 
background/scene audio, thus removing its annotations from the second audio signal. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to modify the combination of Newman and Liaguno which discloses that 
hypermedia can be edited by deleting/removing data (video, text and audio etc), where 
the image data is converted/searchable via text, to form an edited hypermedia medium 
by also deleting the annotations of the second audio signal, in order to provide the user 
the ability to search the desired clips via annotations audio (auxiliary/narrated) or the 
received video. 

In considering claim 3, the claimed utilizing a least-mean square algorithm 
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As discussed above in claim 3, neither Newman nor Cruz disclose removing 
annotations from the second audio signal, nor removing by using a least-mean square 
algorithm. 

The applicant states in the specification, that the least-mean square (LMS) 
algorithm is a conventional algorithm, and also (page 12, line 5-12) that annotations 
may be removed by a variety of methods or algorithms. 

Therefore, it would have been obvious design choice to one of ordinary skill in 
the art at the time of the invention to modify the Newman/Liaguno combination which 
discloses editing and searching hypermedia, by utilizing an algorithm such as a LMS 
algorithm in removing annotations from the second audio signal, in order to provide an 
editing hypermedia which removes undesired audio and allows the user to insert 
additional/auxiliary audio if desired. 

In considering claims 4, 8, 12 and 18, 1) the claimed generating a center text 
title ...2) the claimed generating a scrolling text banner. . . 

Newman discloses a system which uses a non-linear home media editor system, 
which allows the user to develop a audio/video information from either multiple devices, 
to include recorder, VCR, TV, CD-ROM/DVD, digital camera, storyboards, internet, 
etc... Newman also discloses a system which lets the user input the type of 
information, i.e. "OUR SUMMER VACATION", which is displayed in the center of the 
window 408 (Fig 12). Newman also discloses the use of scrolling text overlays, by 
moving prerendered graphical objects onto an image from a variety of hypermedia 
sources (col 12, line 17-23). 
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In considering claims 5, 9, 13 and 19, 
The claimed generated a video abstract via the first and second audio signals, the video 
signals and the searchable annotations is met where the editing hypermedia created, 
generates a video abstract, where the audio signal(s), video signal(s) and other media 
which is included/added/inserted into the unedited hypermedia is combined to form a 
desired hypermedia. As shown in Fig 1 1, the edited hypermedia 'Vacation" (444), is 
broken down into appropriate storyboards for the user which may be edited/viewed. 

In considering claim 14, 
The claimed wherein the video signal is received from a video recorder is met by 
Newman which discloses receiving video via camcorder 586, VCR 588 and digital 
camera 590 (Fig 14). 

In considering claim 15, 
The claimed wherein the first and second audio signals are received from at least one 
microphone is met where Newman discloses the reception of audio/video via camcorder 
586, VCR 588 and also auxiliary audio. 

Conclusion 

4. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure — please refer to newly cited references on attached form PTO- 
892. 
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5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian Yenke whose telephone number is (571)272- 
7359. The examiner work schedule is Monday-Thursday, 0730-1830 hrs. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
Supervisor, John W. Miller, can be reached at (571)272-7352. 
Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

or faxed to: 

(703) 872-9314 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Technology Center 2600 Customer Service 
Office whose telephone number is 
(703)305-HELP. 

General information about patents, trademarks, products and services 
offered by the United States Patent and Trademark Office (USPTO), and other 
related information is available by contacting the USPTO's General Information 
Services Division at: 

800-PTO-9199 or 703-308-HELP 

(FAX) 703-305-7786 

(TDD) 703-305-7785 
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An automated message system is available 7 days a week, 24 hours a 
day providing informational responses to frequently asked questions and the 
ability to order certain documents. Customer service representatives are 
available to answer questions, send materials or connect customers with other 
offices of the USPTO from 8:30 a.m. - 8:00p.m. EST/EDT, Monday-Friday 
excluding federal holidays. 

For other technical patent information needs, the Patent Assistance 
Center can be reached through customer service representatives at the above 
numbers, Monday through Friday (except federal holidays) from 8:30 a.m. to 5:00 
p.m. EST/EDT. 

The Patent Electronic Business Center (EBC) allows USPTO customers to 
retrieve data, check the status of pending actions, and submit information and 
applications. The tools currently available in the Patent EBC are Patent 
Application Information Retrieval (PAIR) and the Electronic Filing System (EFS). 
PAIR (http://pair.uspto.gov) provides customers direct secure access to their own 
patent application status information, as well as to general patent information 
publicly available. EFS allows customers to electronically file patent application 
documents securely via the Internet. EFS is a system for submitting new utility 
patent applications and pre-grant publication submissions in electronic 
publication-ready form. EFS includes software to help customers prepare 
submissions in extensible Markup Language (XML) format and to assemble the 
various parts of the application as an electronic submission package. EFS 
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also allows the submission of Computer Readable Format (CRF) sequence 
listings for pending biotechnology patent applications, which were filed in paper 
form. 




BRIAN P. YENKE 
Primary Examiner 
Art Unit 2614 



B.P. 
28 March 2005 



